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ABSTRACT 

Human error is a crucial problem in manufacturing industries. Due to the misinterpretation of information on 
interface system design, accidents or death may occur at workplace. Lack of human cognition criteria in 
interface system design is also one of the contributions to the failure in using the system effectively. Therefore, 
this paper describes stages of evaluation process on powder handling system prototype. In the study, the 
prototype was evaluated based on cognitive awareness criteria for interface system design: perception, 
comprehension, and projection. Five experts with more than five years of experience from engineering and 
information technology fields were involved in the evaluation process. Results in this study are essential to the 
researcher in order to improve powder handling system prototype. It is hoped that mapping up the cognitive 
awareness criteria in interface system design will help users to make better decisions while handling the system 
at the workplace. 

INTRODUCTION 

Accidents could happen at workplace without warning. This is due to human limitations in handling certain tasks 
that are beyond their control and expectations. In addition, lack of information while using the system is one of 
the results in human error. There are two types of accident implications: permanent disabilities and death. Thus, 
accidents cause a great lost of expertise, time, and money especially for the family, organisation, and community 
as whole. After the World War II, demands for research on complex technical system have increased rapidly. 
Researchers and engineers have started to develop an automated system and focus more on system functions 
without knowing that too many automatic functions embedded on the system cause difficulties for human to 
control the system (Hollnagel and Woods, 2005). 

Then, the research continues on investigating tasks that can be done automatically by a system and tasks that can 
be controlled by human. In relation to this, technology system becomes more complex in order to accommodate 
the role of human as the main user of the system. The ease of use of a system is needed in order to support 
learnability process among users. Ability to understand the system will help users to minimise human error. In 
fact, in line with cognitive research, there is still limited number of researches that focus on interface system 
design as a mediator in system interaction. Therefore, the role of interface system design is crucial in providing 
input for users to make correct judgments in handling the system. 

Hence, this study was conducted with the goal to improve system prototype that included human cognitive 
criteria into interface system design. The prototype was developed in stages and the prototype was improved as 
the evaluation got along. Five experts were involved in the participatory design process starting from the 
prototype sketching until the final powder handling failure configuration functions. In addition, five evaluators 
were sufficient for the discovery of about 75% of the overall system evaluation problems and it was possible to 
achieve substantially better performance by aggregating problems from several evaluators (Preece et. al, 1994). 
Results from this study were crucial in order to improve the powder handling simulation system in accordance to 
increased human awareness in handling the system effectively. 

Cognitive Model 

Classic human cognitive model is also known as Human Information Processing System. The model explains 
how human receives the information from sensory input and transfers it to the brain. The brain will then make an 
interpretation and human will perform an action upon it. 

To make an interpretation, firstly the information from sensory inputs such as visual and auditory information 
will be sent to human working memory. At this stage, if the information is activated by the user regularly, the 
information will be sent to human long term memory. Information stored in long term memory will decay if the 
information remains passive (Friedemberg and Silverman, 2015). To help users to retain their knowledge on a 
particular system, system designers should consider ways to help the users to make use of their knowledge stored 
in the long term memory. 
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Interface system design plays an important role to trigger human long term memory because via interface design, 
user will be able to recognise the information that they used before (Thimbleby, 1998). For that reason, one of 
the solutions is to incorporate the cognitive model into the interface system design. This solution will help users 
to use the system in an effective way. 

Interface design, as an intermediary between users and system plays, an important role for a system. Users will 
perceive information from the interface and interpret the information into meaningful information. Users then 
will act towards related information that is stored in their memory. Sequentially, it is important to include 
cognitive criteria on interface system design because any wrong information conveyed to users via interface 
system design may lead to hazardous condition. Thus, in this study, our aim was to enhance human decision 
making by integrating the cognitive criteria in interface system design. 

Situational Awareness (SA) is commonly defined as perception of the elements in the environment within a 
volume of time and space, the comprehension of their meaning, and the projection of their status in the near 
future (Endsley, 1995). In order to make a wise decision in handling a system, the role of SA in interface system 
design is to provide useable cognitive information for users. In other words, users need to understand available 
information around them because if any unexpected incident happens, users are ready to give immediate 
response. 

Furthermore, recognition-primed decision making research explains that pattern recognition is important in 
decision making (Klein, 1989). Human will select the best alternatives from pattern recognition process rather 
than comparing other alternatives in order to select ultimate solution (Figure 1). For that reason, system 
designers need to consider complementing users’ cognitive requirements in interface system design. 
Additionally, there is interconnected relationship between SA and decision making; if SA is activated, then 
decision making is also activated, and vice versa (Adams, Tenney and Pew, 1995). 



Figure. 1. Cognitive Awareness Model for interface system design. 


Theory Mapping 

Enhancing the prototype interface system design is crucial in mapping up cognitive awareness theory into the 
system design process. In this study, cognitive criteria that will be incorporated in this powder handling system 
were identified through several analyses that we had done earlier (Rosli, 2010; 2011; 2012). Then, the cognitive 
criteria were categorised based on the three levels of SA: perception, comprehension, and projection. Next, as 
shown in Figure 2, the cognitive criteria were mapped up in the powder handling simulation system prototype. 

Perception 

Users perceive useful information from cues in the environment. Stored knowledge in users’ long term memory 
helps them to classify perceived information into meaningful representations. This information is structured in 
accordance with situated time and space. The information is also known as attention-based selection on task 
requirements (Endsley, 1995). Perceived information that matches with the stored information is ready to be 
retrieved by users as a basis for user awareness. 

Affordance is the design aspect of an object that suggests how the object should be used (McGrenere and Ho, 
2000). In other words, affordance provides strong clues to the operation of things. For example, buttons with two 
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layer images (Figure 2) usually give hints to user that the buttons are active and the user can click on the buttons. 
Additionally, by integrating affordances into the interface system design, the user knows what to do by just 
looking at the design, thus instructions, label or pictures are not required (McGrenere and Flo, 2000). 



Figure. 2. Cognitive awareness model mapped with interface system design prototype. 


Furthermore, chunking was also used as a technique to combine many units of information into a limited number 
of units or chunks. According to Miller’s magic number, human can process information in 7±2 items or chunks 
at one time (Banbury and Tremblay, 2004). Thus, short and simple instructions incorporated in the interface 
system design are easy to process and comprehend by users. Chunking information will help users to increase 
their recall performance because structured information is easy to retrieve from their long-term memory. 
Additionally, chunking is used when people are required to recall and retain information. In contrast, chunking is 
not applicable for applications that require searching and scanning information functions. 

Moreover, Gestalt design principles were also used to group buttons with similar functions. For example, the 
navigation buttons are grouped together at the bottom right of the screen, and buttons that represent system 
failure are failed are on the bottom left (Figure 2). Similarities state that things sharing visual characteristics such 
as shape, size, and colour will be perceived as a part of the same form. The eye will easily spot on sections for 
similar functions in order to avoid confusion (Wickens et. al, 2004). 

In addition, to support human mental model in perceiving information in the environment, familiarity, visibility, 
and consistent images and icons will improve the recognition of information that is useful, and help users to 
understand the system. For instance, images that represent an action of an object or concept are meaningful to 
users and require less time to learn. Moreover, this will reduce users’ mental workload and due to that, users will 
be able to focus on more critical tasks (Shneiderman and Plaisant, 2010). 

Comprehension 

At the comprehension level, users will organize and understand the significance of perceived information on a 
particular situation. Moreover, with meaningful interpretation, users will be able to have a mental model about 
the situation stored in their long term memory. At this level, SA is defined as a situational model depicting the 
current state of the mental model (Endsley, 1995). 

In this study, to help users to comprehend information from a system, association cognitive criteria were 
incorporated in the prototype design. For instance, classical conditioning technique was used to associate a 
stimulus with an unconscious physical response (Sobel, 2001). In other words, this technique was to influence 
the appeal of a design with a trigger stimulus, which would evoke an unconscious response. For example, in the 
prototype system design, a blinking red and yellow animated image with high pitch alarm would attract users’ 
attention. In addition, salient cues used in prototype design made the signal visible and noticeable by users (Dix 
et. al, 2004). 

Moreover, to aid users with better understanding in handling the system, information displayed on the interface 
system design needs to be semantically associated together. For example, in the prototype system simulation in 
this study, if the user clicked the “play” button, the powder will be transferred from the container to Silo tanks. 
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An indicator will move up from its original position to show that the powder was filled up into the Silo tank. 
Once the Silo tank was full, the tank will change colour to inform users that the tank had been loaded with 
powder chemical. The same general concept applies when we fill up a bottle with water. By including general 
and informative information in the design, it is hoped that users will be able to enhance their leamability process 
in using the system. 

Projection 

At the final stage of SA, projection is achieved via knowledge of the status and the dynamics of the elements and 
comprehension of the situation (Endsley, 1995). The mental model helps users to understand the situation and 
allows them to generate probable solutions to deal with future states of the system. 

Feedback projection criteria included in system design is important to guide people in using the system. 
Feedback is crucial as the system will send back information about the next actions that users they need to take. 
For example, in system design, feedback can be observed visually or auditorily. Furthermore, to include 
feedback cognitive criteria in interface system design, feedback should provide direct and simple feedback that 
users can understand. 

Additionally, pop out messages and signal display incorporated in the system will help users to make a wise 
prediction on actions that they need to take. Therefore, it is important to design a system that is meaningful so 
that users will understand the overall concept of the system that they use in their daily working tasks. For that 
reason, in this study, we included three common problems in system maintenance, which are motor, cable 
position, and driver failure problems. In relation to this, short messages on system failure configuration solutions 
will be displayed upon request by users. Therefore, it is hoped that by mapping up the cognitive criteria in the 
interface system design will help users to understand and perform well while interacting with the system.. 

Prototype Development 

Prototype evaluation process is essential to seek comments or ideas in improving the final outcome of a system. 
The activity done in developing a prototype encourages reflection in design (Schon, 1983). In fact, the activity is 
an important feature in design process. The aim of having a prototype evaluation is to move through diverse 
design ideas until the idea that meets user requirements has finally been identified. 

In this study, the prototype evaluation ran in stages. The four basic steps were designing, developing, testing, and 
analysing. For each stage of the prototype evaluation process, participants were asked to explore the prototype. 
No specific time was allocated for them to explore the prototype. Once they were satisfied with the prototype, 
they were then to answer the prototype checklist given to them. Then, feedback from the participants were 
collected and analysed in order to retrieve ideas to improve the system design. Next, the suggestions were 
incorporated into the next stage of prototype development process. This process was complete when all of the 
system requirements were fulfilled. 

The evaluation began from low fidelity prototype to high fidelity prototype. For instance, in this study, the low 
fidelity prototype included prototype storyboards of the interface system design. In fact, the low fidelity 
prototype storyboards consisted of a graphical representation of the real system design without any actual system 
functioning (Dix et. al, 2004). Since the low fidelity prototype highlighted only the layout of the system, the 
prototype checklist given to them only covered the cognitive awareness perception criteria (Table 1). In contrast 
with high fidelity prototype, the prototype included interactive screens with the final interface system design. At 
this stage, the prototype checklist given to the participants consisted of cognitive criteria literally related to 
perception, comprehension, and projection. 


Table 1: Stages of prototype evaluation process. 


Prototype 

Cognitive Awareness Criteria 

Low fidelity prototype 1 

Perception 

Low fidelity prototype 2 

Perception 

High fidelity prototype 3 

Perception, Projection, Comprehension 

High fidelity prototype 4 

Perception, Projection, Comprehension 

High fidelity prototype 5 

Perception, Projection, Comprehension 


Participatory Design Evaluation Process 

Participatory design is an evaluation design that covers overall prototype development process. Participants 
whom are usually experts in the work context are encouraged to be actively involved in the design process in 
order to improve the system design. In this study, participatory design was employed for a few reasons. First, the 
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participatory design allowed the participants to contribute directly in the system development stages. Second, the 
participatory design was able to enhance the interaction between the system and working environment. Third, the 
participatory design was used to evaluate and refine the system design at each prototype development stage. 

In this study, five experts from Engineering and Information Technology (IT) background were involved in 
improving the prototype interface system design. At this stage, two engineering experts who had experience in 
system maintenance contributed in improving the interface and functions of the system and the other three 
experts were from the Technical Education and IT field who concentrated on the interface and interaction of the 
prototype. To gather ideas and comments from the participants, a prototype checklist was given to each of them. 
There was no restriction of time for them to complete the prototype checklist, but they were given at least 2 
weeks to evaluate the prototype. Then, the checklist was collected from the participants for further analysis. 


Table 2: Participants’ background 


Participant 

Designation 

Years of 
service 

Expert’s role 

PI 

Instrumentation 

Engineer 

>10 years 

Interface and system 
functions 

P2 

Maintenance Engineer 

>5 years 

Interface and system 
functions 

P3 

Technical Education 
Instructor 

>5 years 

Interface and interaction 

P5 

IT Instructor 

>10 years 

Interface and interaction 

P6 

IT Instructor 

>10 years 

Interface and interaction 


The prototype checklist used in this prototype evaluation process consisted of a checklist and open-ended 
questions. During the evaluation process, participants were encouraged to give comments and ideas related to the 
prototype interface system design. Participants were also free to ask questions to the researcher should they 
required further explanation about the system or any other problems that they encountered during the evaluation 
process. 

The prototype checklist used in this study consisted of three levels of interface design cognitive awareness 
criteria, which were analysed from researcher’s related studies. Most of the criteria were analysed from cognitive 
theories, interface design principles, interaction design, cognitive method studies, and ISO system design 
principles (Rosli, 2012). 

FINDINGS AND DISCUSSION 

Comments and ideas from the participants were essential at this stage to improve the final prototype interface 
system design. One of the challenges in participatory design in prototype evaluation process was time constraint. 
Generally, it took more than six months for the researcher to develop the prototype, to seek feedback from 
experts, and to improve the prototype until the experts were satisfied with the prototype. Findings from each 
level of the prototype were organised in a table form Feedbacks from the participants are discussed in the 
following sections. 

Findings on Perception 

At each stage of the prototype evaluation process, the prototype was improved based on inputs from the 
participants. In relation to visibility criteria, almost all the participants understood the information displayed on 
the system at all stages of the prototypes (Figure 3). For example, three basic saturated colours were used in the 
prototype design due to the fact that saturated colours were able to attract user attention. Even to date red and 
yellow colours are used for alarm signal because they resemble alarm signal in normal daily life. The red- 
coloured signal represents danger while yellow-coloured signal alerts people to always get ready for any 
circumstances that could occur. Moreover, red is commonly used to show danger signal because the colour has 
the longest wavelength and therefore it can be seen from afar (Forsyth and Ponce, 2011). 

Next, similar to visibility cognitive criteria, almost all of the participants were familiar with the functions of 
powder handling system. For instance, they knew that the system begins with first stage of low fidelity prototype 
and ends with final stage of high fidelity prototype. However, three participants claimed that the system design 
should help them to recall any knowledge about system technology that has been stored in their long term 
memory. Two participants highlighted that, indication signal showing the Silo tank was filled with powder was 
rather small. In order to improve the design, the size of the indicator should be bigger. Besides, the participants 
also suggested for the Silo tank to be changed to other colour to indicate that the Silo tank was full with the 
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chemical. As a matter of fact, visible information is vital in interface system design as it helps users to recall 
information that they have experienced before (Chance, 2008). 

Moreover, for chunking, consistency, and affordance criteria, almost all the participants reported that the 
cognitive criteria were properly integrated in the system starting from prototype stage four. Therefore, focus 
should be given to these three criteria in order to design a system that could help to enhance human performance. 
Instructions and solutions should be designed in short and simple words so that designer can chunk the words, 
highlight important words, and bold or colour the text. If the sentences are too long, designer can use bulleted 
form or break the sentences into shorter sentences. On top of that, less time will be used by users to find useful 
information while interacting with the system. 

As for affordance criteria, system designer can design the button with big to small or bright to dim button style if 
their concern is it to attract user attention in using the system. In the study, the participants also claimed that the 
prototype was yet to be consistent after stage three because only one or two buttons were linked to the correct 
page. It is important that to retain users’ interest to the system, thus the system should be free from broken 
linkages. Other than that, the participants also stated that it was not necessary to arrange the buttons according to 
frequency of use, i.e., most frequently used button is at the bottom right corner while the least used button is on 
the left. 

Next, in terms of system prototype layout, the participants were satisfied with the system layout as the simulation 
area was designed in the middle of the system’s screen and the buttons were arranged at the bottom. In point of 
fact, users will be more focused on the simulation if it is in line with human focal point (Wickens et. al, 2004). 
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Figure. 3. Perception design criteria. 


Findings on Comprehension 

Comprehension and projection criteria were only evaluated by the participants starting from high fidelity 
prototype type because stage I and stage II of the prototype only concentrated on ways users perceived the 
system. Therefore, as shown in Figure 4, almost all the participants agreed that generalisation characteristic in 
comprehension criteria should be improved continuously until the final stage of prototype. 
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Figure. 4. Comprehension design criteria. 


A system that is easy to use should provide functions that are general so that the system is applicable for users of 
all levels such as the novice, intermediate, and expert users. For instance, options to display or hide labelling for 
the prototype system design is included in the prototype design. The system labelling is useful for the novice 
users. On the other hand, expert users will only retrieve the information if they find that the system labelling is 
needed. Moreover, it is important to design a system that fulfils user requirements, so that users will be able to 
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comprehend information conveyed to them via the interface system design. 

The participants also agreed on the signals that used daily life routine in association with system design concept 
(Thimbleby, 1998). In conjunction with learnability and informative criteria, the participants seemed to 
understand the short notes provided at the top left corner of the systems. The short notes explained the impact if 
one of the system components broke down. The short notes also helped the participants to recall possible 
consequences that could happen to lead them think of the solutions to overcome any arising problems (Preece, 
Rogers and Sharp, 2006). To optimise safety at workplace, users will always be ready to face any hazardous 
situations as long as cues or information is visible and able to capture their attention. Thus, users can at least 
immediately think of the solutions to safe one’s life. 

Findings on Projection 

It is a challenge to integrate projection design criteria in the prototype interface system design. This is because 
system designer needs to develop a system that can guide users to give positive response while using the system. 
In this study, the participants showed that an improvement was still needed even at the final stage of the 
prototype system evaluation to make sure that users can deal with the information conveyed to them accordingly 
(Figure 5). 
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Figure. 5. Projection design criteria. 


For instance, as reported by the participants, they found it hard to predict the type of system fault that could 
occur in the prototype. Participants suggested providing a short message informing users on types of system fault 
that could happen at any time. Providing such information will help the users to analyse the problems and for 
them to come up with possible solutions (Dix et. al, 2004). 

In order to give meaningful simulation, two participants suggested that symbols or images used in the system 
design should use engineering standard symbols for system or machines in order to avoid confusion among 
users. Unambiguous symbols in the system will help users to have a clearer picture on how the system works 
(Norman, 1986). 

Feedback, guidelines, solutions, short notes, and pop up messages will help users to have a quick analysis on the 
next actions that they need to take in dealing with the system. In this study, all the participants were satisfied 
with the concept of continuity embedded into the system. By highlighting important information at the earlier 
stage of the simulation process, it is able to attract user attention to the system. As people tend to lose focus at 
the end of a particular situation, important information should be highlighted at the earlier stage of system design 
(Chance, 2008). 

Additionally, to represent similar meaning and command to user mental model, designer can combine the 
information in displaying warning to attract user’s attention, integrate the information in designing signage, or 
incorporate the information in logo to make the identification process faster. As a whole, in this study, the 
participants showed rising growth of agreement towards the end of the final stage of prototype development 
process. The participants were able to use the system smoothly and the incorporated cognitive criteria helped 
them in handling the system.. 

CONCLUSIONS 

This paper describes the importance of cognitive awareness criteria incorporated in the prototype interface 
system design. The prototype was evaluated in stages by the participants in order to enhance the interface system 
design. Ideas and comments from the participants were essential to the researcher as these ideas and comments 
were useful in improving the powder handling simulation system for further research analysis. 
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In general, it was rather a challenging task to incorporate the cognitive awareness criteria in the prototype system 
design. Progress in developing the prototype was expanding moderately and the prototype was improved after 
each stage of evaluation process. From the perception criteria feedback, three criteria that have to be given more 
time to design in designing a system were affordance, consistency, and visibility criteria. System users need 
information that is able to evoke their memory so that they can select and react accordingly while handling the 
system. Similar to consistency, it is important to keep the design consistent in order to avoid ambiguous feeling 
to users while they are interacting with the system. 

As for comprehension criteria, a system that supports users’ learnability process allows users to own the skills in 
controlling the system. If problems occur, they will be able to act automatically and think precisely on the 
actions that they need to take while handling the situations. It is a challenge to provide information that meets 
users’ cognitive requirements. Therefore, active contributions from the participants in designing the system were 
essential to make sure that users’ cognitive needs were incorporated in interface system design. 

In order to help users to perform well in their decision making, system designer needs to focus more on ways to 
design feedback and meaningful projection criteria in system design. Due to human abstract thinking, system 
designer also needs to design a system that helps users to understand the system and situations around them. 
Appropriate and useful information are thus crucial to system users as the information will be observed and 
analysed by them in order to select the best actions while interacting with the system. It is hoped that cognitive 
criteria will help users to enhance their performance in using the system and help them to minimise human error 
at workplace. 
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